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(57) Abstract: A system allows large numbers of users to engage in on-line, multi-user shows that are synchronized to broadcasts 
of prerecorded or live shows and also simultaneously aggregates user input from such shows and feeds it back into the broadcast 
First, a user launches client application on an Internet-accessible platform. The client application connects to a multi-user server 
system. This multi-user server system allows all of the concurrent users to engage in a single multi-user, on-line activity that can 
run independently or in synchronization with a broadcast The multi-user server system is also capable of ninning numerous on-line 
activities simultaneously, each of which can be synchronized to a separate broadcast As the users interact with die on-line activities, 
the multi-user server system can aggregate data relating to the users or their actions, save this data in a database, and feed this back 
into the broadcast The multi-user server system also includes the ability to send messages to all the users or specific groups of users, 
adjust the synchronization of the on-line shows, schedule on-line shows, and create scripts for running the shows in synchronization 
with the broadcasts. 
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SYSTEM AND METHOD FOR SYNCHRONIZING ONLINE ACTIVITIES 
WITH BROADCAST PROGRAMMING 

BACKGROUND OF THE INVENTION 

5 1. Field of the Invention 

The present invention relates to systems for synchronizing broadcast media 
transmissions with other forms of communication. More specifically, it relates to systems which 
allow a large number of broadcast media recipients to engage in on-line activities that are 
synchronized to transmissions of the broadcast medium - television, radio, Internet or any other 

10 type of broadcasts, while simultaneously aggregating user input and feeding it back into the 
broadcast transmission. 

2. Background of the Related Art 

Radio and television broadcasting have revolutionized the way that information and 

1 5 entertainment are distributed in the 20 th century and have become truly ubiquitous. Nearing the end 
of the 20 th century, the development of the Internet and the World Wide Web have provided the 
opportunity to bring a new, powerful technology for providing information and entertainment via a 
worldwide network of computers to tens of millions of users. 

By tightly integrating the user experience of broadcasting and the Internet, it would 

20 be possible to create new shows and content that would greatly extend the capabilities of both 
media. Tightly coordinated on-line / on-air programming could be used for a wide variety of 
content types, including game shows, news / opinion shows, sports, entertainment, education 
shows, children's programming, and advertisements. Additionally, if such integration could 
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support the simultaneous coordinated activities of a very large numbers of users, it would further 
extend the creative possibilities and support the development of robust on-line / on-air 
communities. Finally, this system could be capable of driving real-time information back into the 
broadcast in order to "close the loop" and fully integrate the coordinated user experience. 
5 However, there is a challenge imposed by such an environment The deployment of 

systems using enhanced set-top boxes and cable system head-end equipment is delayed by a 
fractious standards battle and the reluctance of broadcasters and cable system operators to invest in 
potentially soon-to-be obsolete technology. Meanwhile, there is a huge pent-up demand for 
combined Internet / broadcast programming, as evidenced by the millions of users who 
10 simultaneously access the Internet on a personal computer while watching television (estimated by 
one study to be 1 8 million households) without any content or shows being created to support that 
activity. 

A system which uses innovative software technology and content delivery methods 
is needed to achieve the tight broadcast / Internet integration so many users desire. This system 

1 5 must do so in a manner that allows rapid deployment as well as adaptability to future developments 
in cable television and broadband delivery systems. 

This system should achieve its tight coordination with broadcasts at the broadcast 
control and Internet server level in order to avoid the fractured and technically uncertain 
environment presented by attempting to deploy content using other cable television, satellite or 

20 broadband-based technologies. The client application design should support being deployed on a 
wide range of client devices, from current personal computers to devices not yet built. The client / 
server design should provide a tight synchronization for the users* on-line / on-air experience no 
matter what variations they may experience in network latency and bandwidth. Multi-user server 
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capabilities must be included to support the simultaneous response of very large numbers of users 
in coordination with the broadcasts. The server system should be able to aggregate, track, archive 
and report on the simultaneous multi-user activities while storing the data and user profiles in a 
database. In addition, the system should drive real-time information and graphics back into the 
5 . broadcast to fully complete the "loop" for the users* on-line / on-air experience. Finally, this 
system must be robust, reliable and scalable to meet the real-world demands of broadcasters. 

SUMMARY OF THE INVENTION 
The present invention has been made with the above problems of the prior art in 
1 0 mind, and it is an object of the present invention to provide a method and system by which large 
numbers of users engage in on-line, multi-user shows that are synchronized to television, radio, 
Internet or any other type of broadcasts, while simultaneously aggregating user input and feeding it 
back into the broadcast. 

According to a first aspect of the present invention, this object is achieved by 
1 5 providing a system in which a user first launches a client application which is either downloaded 
from the Internet or pre-installed on the user's Internet access device. The client application is 
based on a set of Application Programming Interface (API) modules which are created as part of 
the system and designed to be easily deployed on a wide range of Internet access devices. These 
devices can be Internet-enabled personal computers, enhanced set-top box devices, wireless 
20 devices, Internet access appliances or any other devices that support a programming language and 
standard Internet communication protocols. 

Next, the client application connects to a multi-user server system. This multi-user 
server system allows all of the concurrent users to engage in a single multi-user, on-line activity 
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that can run independently or in synchronization with a broadcast. The multi-user server system is 
also capable of running numerous on-line activities simultaneously, each of which can be 
synchronized to a separate broadcast. These on-line activities can be integrated with virtually every 
type of content, both pre-recorded and live shows, including but not limited to news / opinion 
5 shows, sports, drama, comedies, game shows and advertisements. The on-line activities include all 
types of multi-user and single-user applications, including, but not limited to, chat, games, opinion 
polling, interactive commercials, and e-commerce. The client / server system provides a tight 
synchronization for the on-line / on-air content, which accommodates variations in the Internet 
network's and user's connection latency and bandwidth. 

10 As the users interact with the on-line activities, the multi-user server system will 

aggregate data relating to the users and/or their actions, save this in a database and feed relevant 
data back into the broadcast as graphics or other forms of information. The multi-user server 
system also includes the ability to send messages to all users or specific groups of users, adjust the 
synchronization of the on-line shows, schedule on-line shows, and create scripts for running the 

15 shows in synchronization with the broadcasts. The multi-user server system is deployed in a 
primary / secondary server configuration, where the loads are distributed across multiple server 
computers. This helps to make the system easily scalable for expansion to accommodate 
increasing user numbers. The system preferably provides a control application so that broadcast 
control room personnel may manage the combined on-line / on-air content. Additionally, the 

20 system provides broadcasters tools for creating the on-line / on-air content, specifically a 
scheduling application and a show scripting application. 
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BRIEF DESCRIPTION OF THE DRAWINGS 
These and other objects, features and advantages of the present invention are better 
understood by reading the following detailed description of the preferred embodiment, taken in 
conjunction with the accompanying drawings, in which: 

FIG. 1 shows an overview of a presently preferred embodiment of the present invention; 
FIG. 2 is a flow diagram showing how the embodiment functions; 
FIG. 3 is a diagram showing various client devices usable in the preferred embodiment; 
FIG. 4 is a block diagram showing the server system configuration in the embodiment; and 
FIG. 5 is a time-based chart depicting the sync-to-broadcast system flow in the 
embodiment. 

DETAILED DESCRIPTION OF 
PRESENTLY PREFERRED EXEMPLARY EMBODIMENTS 
The present invention provides, in a preferred embodiment, a system that allows 
1 5 large numbers of users to engage in on-line multi-user activities that are tightly integrated and 
synchronized with television, radio, Internet or other types of broadcasts, while simultaneously 
aggregating user input and feeding real-time information pertaining thereto back into the broadcast. 
This is deployed in a client / server environment. The system achieves its tight coordination with 
broadcasts at the broadcast control / Internet server level in order to maximize control of the 
20 synchronization and integration and avoid depending upon downstream technologies which may or 
may not be available to all users. 

FIG. 1 provides an overview of the preferred embodiment, illustrating how all the 
components work together to synchronize multi-user on-line activities to a broadcast. Here, a 
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broadcaster 101 transmits a broadcast signal 102 which may involve a television broadcast, radio, 
the Internet or any other type of broadcast. Although in the following discussion reference will be 
made to television broadcasts, this is in no way limiting, and the transmission may be via cable, 
airwaves, network or any other type of transmission medium. 
5 Audience households 1 03 receive the broadcast 1 02 via an appropriate device such 

as a satellite dish, antenna, cable, Internet connection or the like - in fact, virtually any network 
capable of simultaneously delivering streaming content to multiple users can be used. For 
networks which provide effectively instantaneous delivery of content, no provisions need to be 
made for latency in transmission time; however, for those which do exhibit transmission latency 
10 (such as the Internet), compensation may be appropriate as will become apparent to those skilled in 
the art. 

These households are connected to the Internet 104 or similar communications 
network. A typical household 106 contains a viewer/user 108, a device 107 capable of receiving 
the broadcast signal 102, e.g., a television, radio, computer or other appropriate type of receiving 

15 device, and an Internet-enabled client device 109. Preferably, the Internet-enabled client device 

109 supports a language like C++ or Java for executing a client application 412 (see Fig. 4) resident 
thereon as well as a standard Internet protocol, and is connected to the Internet 104 so that the 
viewer 108 can interact with the client application 412 while viewing the receiving device 107. 
This client device 109 can either be integrated into the receiver device 107 (e.g., an enhanced TV) 

20 or exist as a stand-alone device (e.g., a personal computer or wireless device). Alternatively, the 
client device 109 can share a screen with the receiver device 107 (e.g., a WebTV or set-top box) or 
have its own separate screen as shown in FIG. 1 . In the following, reference will be made to a 
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personal computer as the client device 109; however, this is only for the purposes of explanation, 
and other equivalent devices as described above may also be used. 

The client application 412 that runs on the client device 109 stays in synchronization 
with the broadcast 102 by communicating with the multi-user server system 111, described in 
5 greater detail below. The client application 412 receives scripts outlining the flow of events from 
the multi-user server system 1 1 1 and runs the on-line shows and activities according to the timeline 
laid out in these scripts. It also receives synchronization cues from the control application 1 12 and 
server system 1 1 1 (also described in greater detail below), thus keeping the script in 
synchronization with the broadcast 102. 
10 The scripting language is preferably time code-based. The time code may reside on 

a video tape, audio tape or any other medium. Every event in the activity is pegged to a specific 
time code, thereby creating a timeline made up of a sequence of key events. These events may 
include, but are not limited to, sending messages to users 108; displaying graphics, playing 
animation and playing sounds; delivering advertising content; coordinating interactive actions; and 
1 5 receiving input from the users 108. 

For example, a show may start at a time code of 1 :00:00:05. This might represent a 
position of five video frames from the start of a video tape. If the show were a trivia-based game 
show, the first question could be queued at 01 :00:30:05. This would represent an action taking 
place thirty seconds into the show. In this way, online actions can be synchronized to the time code 
20 of the source broadcast medium. 

In addition, the client application 412 preferably sends back time-stamped data 
detailing when specific actions take place locally, i.e., within household 106. There are two types 
of data reported to the multi-user server system 111: user-initiated data, this data may come from 
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text entries, chat, responses, instant messaging, etc.; and tracking data which relates to the status of 
the client application. Data of the first type may include, but are not limited to, answering 
questions, sending text messages, entering data, and interacting with other on-screen multi-media 
elements. Preferably, this information is encrypted to prevent alteration by the users 108 or other 
5 parties. Further, time stamping the data accommodates variations in latency and bandwidth across 
the user base 103. Also, it is preferable that the local action reports includes a portion specifying 
the length and type of report, as well as a sequence number which is used by the multi-user server 
system 1 1 1 to ensure that reports are not taken out of order. 

In addition to transmission of such above deliberate activities of the a user 108, the 
10 client application 412 preferably also sends back data gathered from other user interactions as data 
of the second type. These interactions may include data gathered from the user's connection to the 
server 111, e.g., the activity in which the user 108 is involved; the advertisements the user 108 has 
seen; where the user 108 goes within the system; and when the user 108 logs on and off of the 
system. 

15 This data is preferably aggregated and stored in a database 405 residing on the 

multi-user server system 1 1 1 as shown in FIG. 4. This database is used for archiving and tracking 
information associated with the on-line activities, including user profiles, interaction, scoring 
results, ad views and other data types. In the archiving process, it is preferably that every user 108 
has a unique user profile, and every session in which a user logs on has a unique ID code; thus, data 

20 collected during a particular session can be stored according to a unique user ID and session ID. 
Additionally, the date and time over which each session occurred is known, so that each session 
can be tracked by any of the above parameters. 

FIG. 2 lays out the sync-to-broadcast system flow. It shows the overall flow of the 
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preferred embodiment, describing how time synchronization information from the original 
broadcast 102 is distributed by the server 1 1 1 to each of the client devices 109, allowing them to 
display program information in synchronization with the broadcast 102. First, shows are broadcast 
normally at Step 201; there is no change in how a radio, Internet or television show is broadcast 
5 Additionally, as shown at Step 202, the television 1 07 or other receiver device received the 
broadcast 102 while requiring no special hardware. 

At Step 203 the client device 109 is preferably in the same room as the television 
107 and the user 1 08 and receives data from the multi-user server 1 1 1 via the Internet 104. The 
client application 412 uses the Internet 104 to establish a real-time socket connection to the multi- 

1 0 user server system 111. From the multi-user server system 111, the client application 4 1 2 receives 
scripts containing synchronization time information, graphics, sound and other assets, as well as 
on-the-fly messaging and synchronization cues. In Step 204, the multi-user server system 1 1 1 
coordinates interaction with all the clients 109, distributes the scripts and assets, directs the flow of 
information, and connects all the clients 109. The multi-user server system 1 1 1 also listens for 

1 5 synchronization cues from the control application 1 1 2 and distributes synchronization time 
information to the connected clients 109. 

In Step 205, the Internet-connected control application 1 12 sends the 
synchronization time information to the multi-user server system 111. This synchronization time 
information may come from either VBI decoder box 1 16 as described above or from a manual 

20 synchronization control. In Step 209, the broadcaster 101 broadcasts the radio, Internet or 
television show 102 normally. In Step 208, the broadcast signal may or may not contain a 
synchronization cue in the VBI. Having a synchronization tone embedded in the VBI by the 
broadcaster 101 is optional; the system can run without any VBI synchronization tones by using the 
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manual synchronization controls, although the VBI synchronization tones help to automate the 
synchronization process. In Step 206, the control application 112 listens for synchronization cues 
from the VBI decoder box 1 16 if necessary, and in Step 207 the VBI decoder box 1 16 decodes 
synchronization cues embedded in the VBI. 
5 In Step 2 1 0, data gathered from the on-line users 1 08 interacting with the client 

application 412 are aggregated by the multi-user server system 1 1 1 and fed back into the broadcast 
102 using the control application 1 12 along with the Chyron 1 13 or similar device. 

The Internet 1 04 sits at the core of this application, and all parts of the preferred 
embodiment communicate over the Internet 104. Although dedicated connections between the 

1 0 multi-user server system 1 1 1 , the control application 1 1 2, the scheduling application 1 1 4 and the 
show scripting application 115 (all described in greater detail herein) could be used, the design of 
the preferred embodiment eliminates any communication latencies introduced by the Internet 104 
due to the use of a network time protocol (NTP) on all Internet-connected applications. 
Essentially, by transmitting in advance set times to computers which are set by the NTP, actions 

15 can be executed in synchronization even though there might have been latency in transmission of 
the times. Of course, NTP is merely used as an exemplary source of time reference information in 
the preferred embodiment, and other sources, includeing proprietary ones, may be used in its place. 

A messaging API for common client / server functions facilitates development of 
the client application 412 on the client device 109. The messaging API covers over fifty integral 

20 commands including, but not limited to, "enter lobby", "request available events", "request 

available rooms", "create new room(s)", "enter room", "exit room", "request user list for room", 
"request and edit beeper list", "initiate or respond to private messages". Specific to the 
synchronization there are commands such as "start event", "end event", "kill event", "speed up 
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event", "slow down event", "end event" and the like. These commands are accurate to 
milliseconds. This messaging API is fully extensible for the development of further features. This 
API utilizes sockets so it is not dependent on any particular language or schema. Any developer 
language or front end that can interact with sockets can effectively work with the messaging API. 
5 The developer is free to display or not display the information retrieved from the messaging API as 
suits their product. 

Returning to FIG. 1, the multi-user server system 1 1 1 coordinates and manages the 
entire process. It provides a true multi-user platform for allowing numerous simultaneous users 
108 to interact with one another and with the on-line show. It synchronizes the client devices 109 

10 in all households 1 03 with the server 111. It works in conjunction with the control application 1 1 2, 
scheduling application 1 14, and show scripting application 1 15 (all described in greater detail 
below) to control the synchronization, schedule the on-line shows, send out updated assets such as 
sound, graphics, video and text files, deliver advertisements, send new script files, and provide and 
coordinate the various media elements. It preferably gathers time-stamped data from each of the 

15 clients 109 and stores this data in a database. It also preferably aggregates other data from the users 
108 and stores this data in a database. It makes data in the database available to the control 
application 1 12 for integration back into the broadcast 102. 

The control application 1 12 schedules the run time / date for both the sync-to- 
broadcast shows and activities as well as the on-line non-synchronized shows and activities. It is 

20 preferably comprised of the following: 

- an optional automatic synchronization control which uses a VBI (V ertical 
Blanking Interval) decoder box 1 16 to listen for a tone imbedded in the VBI to synchronize the on- 
line show with the broadcast 102; 
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-- a manual synchronization control for synchronizing the on-line show with the 
broadcast 102, e.g., a button in the control application 112 which can be pressed when the 
broadcast show starts; 

- a response data control for managing user activity response data information; 

a data integration control for integrating information gathered from the users 108 
back into the live television, radio or Internet broadcast 102 for example, determining the highest 
scoring users 108 in a broadcast competition and feeding those scores back into the television 
broadcast using the control application 1 12 which can gather the high scores from the database 405 
of the multi-user server system 1 1 1 and display them on a computer screen, the image then being 
captured using a Chyron 113 and integrated into the broadcast signal 102; 

- a broadcast message control for sending a real-time message to the users 108, e.g., 
a message that the show is delayed, that a new show will air next week, or that a new prize is being 
offered; 

- a server status control for displaying the status of the server 1 1 1 to the system 

operator; 

- a start / stop control for manually starting and stopping the on-line show, thereby 
allowing the operator of the control application 1 12 to terminate a show due to technical 
difficulties, scheduling problems and the like; 

- a slip synchronization control for manually shifting the synchronization forward 
or backward by a number of milliseconds, useful because manual synchronization control may be 
inaccurate or the broadcast is faulty and VBI information does not start online activities at the 
correct time; 

- a kill control for terminating a specific function within an on-line show; and 
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— a run reports control for generating reports on usage, user profiles, scores, 
response data, advertising and other information present in the database. 

A Chyron 1 13 or a device similar in functionality, i.e., one that can place graphics or 
text in a television display, is used to integrate data back into the live television, radio or Internet 
5 broadcast 102. 

The scheduling application 1 14 provides a tool for scheduling the time / date of both 
on-line sync-to-broadcast shows and on-line non-synchronized shows. The scheduling application 
1 14 essentially sets up a calendar in which dates and times of activities corresponding to events in 
on-air broadcasts are entered. In advance of these times, corresponding scripts are assembled and 

10 sent to clients 109 to run in synchronization with broadcasts 102. This system can easily be used 
with non-synchronized activities as well as sync-to-broadcast shows. 

The show scripting application 1 1 5 is used for creating controlling scripts for on- 
line sync-to-broadcast shows. It is comprised of the following: a script generator for matching 
specific on-line actions with time code from video tapes; an asset integration tool for adding text, 

15 artwork, sounds and music to the on-line shows; an editor for modifying the scripts and assets; and 
an uploading routine for saving the scripts on the multi-user server system 111. The use of the 
scripting application for videotaped shows will be readily apparent to those skilled in the art. For 
live shows, rough synchronization may be obtained if general event timings are known, or 
sequences of events, e.g., in the case of a football game, can be predicted. 

20 The VBI decoder box 1 1 6 decodes synchronization cues embedded in the VBI. 

Preferably, the control application 1 12, scheduling application 1 14, show scripting application 1 15, 
client application 412, and multi-user server system 1 1 1 all communicate via the Internet 104. 
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As shown in FIG. 3, the multi-user server system can work with a variety of 
different devices, as long as these devices are Internet enabled, support standard Internet protocol 
and support a programming language such as C++ or Java. The first component shown is a 
receiver device 1 07. This can be a television, radio, computer or some other device that can 
5 receive the broadcast signal 102. A user 108 views the broadcast and interacts with the client 
application 412 resident on the client 109. 

The multi-user server system 111 can communicate with the client application 412, 
which can be deployed on a wide variety of devices as shown by devices 109 and 109a-109d. One 
possible device 109 is an Internet enabled PC running Windows or another operating system. This 

10 device communicates with the multi-user server system 1 1 1 over the Internet 104. Another 

possible device 109a is a WebTV-type device that supports a programming language and standard 
Internet protocol. Another possible device 109b is an enhanced set-top box that supports a 
programming language and standard Internet protocol. Yet another possible device 109c is a 
wireless unit that supports a programming language and standard Internet protocol. Still another 

1 5 possible device 1 09d is an enhanced TV, digital television or radio that supports a programming 
language and standard Internet protocol. Additionally, although not shown in FIG. 3, other devices 
may be used, such as a cellular telephone, personal digital assistant, tablet device or any other 
Internet connected device which supports a programming langauge and standard Internet protocol. 
All of these devices 1 09 and 1 09a- 1 09d can communicate with the multi-user server system 1 1 1 

20 through the Internet 1 04. Basically, any client platform that supports sockets can be used to 
develop sync-to-broadcast activities. 

As shown in FIG. 4, the multi-user system server 1 1 1 preferably has two main 
components: the primary server 402 and the secondary server or servers 407. The primary server 
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402 accepts users, validates them (preferably using a user ID/password scheme or similar technique 
as is known in the art) to ensure that the users 108 are registered and are connecting from the 
proper client 109, and handles the process of moving them from room to room where various 
activities, e.g., chat, games, etc., take place. The secondary server 407 is responsible for 
5 monitoring a set of rooms that are all on that server. When the load manager application 403 
assigns a user to a room, it hands that user off to an activity manager application 408 . When an 
activity is over, the activity manager application 408 reports any outcome information to the load 
manager application 403. There is a separate activity manager application 408 for each server 407. 

The client application 412 makes its initial connection over the Internet 104 with the 

10 load manager application 403. The load manager application 403 deals with the specific problem 
of managing large numbers of users and activities scaled across multiple servers 407 and the like. 
It communicates with other server components and client applications 412 using TCP/IP. The load 
manager application 403 resides on the primary server 402 or servers. When the load manager 
application 403 initially launches, it queries the system database 405 for scheduled events and pre- 

1 5 allocated rooms. When it has received this data, it listens for each of the activity manager 

applications 408 to connect. Upon completion, each activity manager application 408 is told which 
events and rooms to load. By querying the database, the load manager application knows which 
activities - trivia games, puzzles, chat, opinion polls, user feedback sessions and the like ~ are 
present in which rooms and how to route users to and from those rooms. 

20 The load manager application 403 authenticates the user 108 by accessing the user 

database 405. When a user 1 08 has been validated (and preferably information on which activities 
the user's client 109 can support is obtained therefrom), they are placed into the lobby. The lobby 
is a place where users 108 can choose to enter or be assigned to rooms containing different 
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activities. Its primary purpose is to enter users into the system and move them into activity rooms. 

Users 108 in the lobby preferably may be asked to choose from a dynamically 
changing list of available events and rooms, or they may be placed automatically into rooms 
assigned by the load manager application 403. Once they have either selected an available room or 
5 been assigned one, the load manager application 403 places them in the room and, at this point, 
they become the responsibility of the activity manager application 408. Users may re-enter the 
lobby in order to move to another activity room. 

Network time protocols servers 406 and 410 provide critical time synchronization 
information to all software components; in the preferred embodiment, these are NTP servers, 

10 although as noted above other synchronization information sources may alternatively be used. On 
the primary server 402 side, a primary NTP routine 406 provides the initial time synchronization 
which is coordinated with a source of time reference information. This primary time 
synchronization is accessed by the activity manager applications in each of the secondary servers 
407 in the system. Accuracy is in the milliseconds within the system. As each client 109 logs onto 

1 5 the system, they too access the time synchronization information. Accuracy of this time 

synchronization over the Internet 104 ranges from the tens of milliseconds to a second. All events 
precipitated by the server are cued relative to their NTP time, and are displayed by the clients 
according to their NTP time. 

For example, when the initial cue to start a segment is received, this cue is 

20 "stamped" with the NTP time at which it was received. When this cue is distributed to the client 
machines 1 09, they can then calculate when to display the cues within the segment relative to their 
own NTP time. In this way, it is possible to retain synchronization between the server 1 1 1 and 
each connected client application 412 over the course of the synchronized event without any clients 
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109 being hindered by latency. Periodic checks against the server's NTP time are made to keep 
client machines 109 synchronized within one second of each other and the server 111. 

In this way, the NTP is used to provide synchronization information over the 
Internet, rather than over a dedicated LAN as is customarily the case. It is believed that this NTP 
5 Internet synchronization technique has not been practiced in the prior art and is unique to the 
present invention. Also, as noted above, although NTP is used in this presently preferred 
embodiment, other sources of synchronization information may alternatively be used, including 
proprietary or semi-dedicated sources. 

Activity modules 409 are the client/server software pieces that the user 108 interacts 

10 with during the course of an event These modules are loaded and unloaded based on commands 
from the load manager 403. For example, the load manager 403 may instruct the multi-user server 
system 1 1 1 to load a trivia game activity into a number of rooms and make those rooms available at 
3 p.m. to anyone logging into the system from a specific client application 412 associated with a 
specific broadcast Alternatively, the load manager 403 may instruct the multi-user server system 

15 1 1 1 to unload the trivia games at 4 p.m. and replace them with other activities such as chat sessions 
or puzzles. 

Examples of sync-to-broadcast program modules include, but are not limited to, user 
registration, chat, games, polling, advertisements, e-commerce links, rankings, and announcements. 
Activity modules 409 encapsulate the rules and structure of a single sync-to-broadcast activity or 
20 program, and may do so essentially as hard-coded programs or software modules. 

Media 409 may be stored on either the client 1 09 or the server machines 402 and 
407. In order to reduce bandwidth usage during critical moments, essential media is installed or 
cached on the client machine 109 prior to cue time. Updateable media, such as advertising or new 
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game files, may be streamed either from the multi-user server system 1 1 1 or from external links. 
Web services 404 and 41 1 are used to serve up web pages, script files, assets, client applications, 
and any required media elements or software. Web services 409 and 41 1. may also include an ad 
server for delivering on-line advertising synchronized with the broadcast commercials. 

The communication responsibilities of the load manager application 403 are: 

— start server system 402 with customizable parameters; 

— listen for activity manager applications 408 and ask them to load room-specific 

activity; 

listen for .clients 109 on a specified port; 

— verify clients 1 09 are valid; 

— maintain available room information; 

move clients 109 from room to room as required or requested; 

— receive information from each of the activity managers 408; 

— write information to user database 405; and 

— propagate customizable web updates as scheduled. 

Once all the activity manager applications 408 have connected, the load manager 
application 403 communicates a list of events and rooms to load. When a user 108 selects an event 
and room or is assigned to one, the load manager application 403 passes them to the activity 
manager application 408. Once a user 108 is in a room, their client application 412 speaks directly 
to the activity manager application 408 until they need or request to be placed into another room. 

The activity manager application 408 may be thought of as a conduit between the 
load manager application 403 and the individual clients 109. There is one activity manager 
application 408 for every server in the system except the primary server 402 (although in 
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alternative embodiments, an activity manager 408 may be operative on the primary server 402 as 
well). The responsibilities of the activity manager application 408 are: 

- listen to the load manager application 403; 

- set up threads to handle each activity; dynamically load activity-specific DLLs 

5 (relatively small software programs downloaded to the client application 412 which are designed to 
run applications such as chat, trivia games, etc.); 

- listen for clients 109 and direct them to the proper room; and 

- report information to the load manager application 403. 

Every room has two threads, one thread accepts client 109 into the room, and 
10 another thread controls the room and handles the data exchange back and forth. In addition to 
room threads, the load manager application 403 also maintains threads. The load manager 
application 403 is a server to the clients 109, to the activity manager applications 408, and to the 
control application 1 12. For each of these, the load manager application 403 maintains two 
threads, one to accept new connections, and one to regulate the data among the connections. 
15 The load manager application 403 accesses database 405 preferably using ODBC or 

a similar access scheme to keep track of the following information: user profile; user activity; 
event information; room information; and results tracking such as high scores and usage data. This 
information is valuable to sponsors and advertisers and can also be used to provide data which can 
be used in the on-air component. The activity results are items that can be propagated to the 
20 broadcast community 102 so users 108 can see how they rank. Ranks and other results can also be 
propagated automatically to a customized web page for review. 

FIG. 5 is a time-based chart illustrating the flow of events in a sync-to-broadcast 
show according to the preferred embodiment. First, in Step 501 an on-line show is created using 
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the show scripting application. In Step 502, the on-line show is scheduled using the scheduling 
application. In Step 503, a synchronization tone may or may not be imbedded in the VBI of 
broadcast 102. This is an optional step dependent on the level of cooperation of the broadcaster. In 
Step 504, the broadcast 102 is aired. Using decoder box 1 16, Step 505 detects whether or not a 
5 VBI synchronization tone is present in broadcast 102. In Step 506, if there is a synchronization 
tone embedded in the VBI, the decoder 1 1 6 will pick up the synchronization tone and transmit it to 
the control application 112. In Step 508, if there is no tone embedded in the VBI, the control 
application operator must manually start the sync-to-broadcast on-line show. In Step 507, the 
control application automatically starts the sync-to-broadcast on-line show. 

10 Meanwhile, in Step 509 the user 108 runs the client application 412 on an Internet- 

connected client device 109 while tuning into the broadcast 102. In Step 510, the user 108 logs 
into the system. In Step 51 1, the client application 412 then automatically downloads the 
appropriate script and other assets for the show from the multi-user server 111. In Step 5 1 2, if the 
user 108 logs on before the sync-to-broadcast show has started, the user 108 may be directed to 

15 engage in other related on-line activities. For example, if the on-air/online show has not yet 
started, the user 108 may be placed in a chat room or a game room where they can spend time 
talking to other players before the show is scheduled to start. When the show does start, the 
activity manager 408 may assign a new activity to the room which corresponds to the broadcast, or 
the user 108 will automatically be moved into another room where the appropriate sync-to- 

20 broadcast activity will take place. 

In Step 513, when the sync-to-broadcast show commences, the user 108 is entered 
into a real-time, multi-user sync-to-broadcast on-line show. In Step 514, the client application 412 
follows the show script, keeping in synchronization with the specified timing. In Step 515, to 
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avoid problems with Internet latency and bandwidth variations, the client application 412 time 
stamps local actions. In Step 516, the show script tells the client application 412 to wait and listen 
for additional synchronization commands from the multi-user server system 111. In Step 5 1 7, the 
client application 412 also responds to any manual or on-the-fly commands from the control 
application 112. In Step 518, after the sync-to-broadcast on-line show ends, the user 108 is 
returned to on-line non-synchronized activities. 

The present invention has been described above in connection with a preferred 
embodiment thereof; however, this has been done for purposes of illustration only, and the 
invention is not so limited. Indeed, variations of the invention will be readily apparent to those 
skilled in the art and also fall within the scope of the invention. 



21 



WO 01/39506 PCT/US00/42248 



WHAT IS CLAIMED IS: 

1 . A method of synchronizing on-line content with a broadcast, the 
method comprising: 

transmitting a broadcast signal to a first user reception device via a first 

medium; 

transmitting timing information corresponding to information in the 
broadcast signal to a second user reception device via a second medium; and 

using the second user reception device to generate user-perceptible indicia 
at times corresponding to the timing information. 

2. The method of claim 1, wherein the broadcast signal is one of a . 
television broadcast signal and a cable broadcast signal. 

3. The method of claim 1, wherein the first user reception device is a 

television. 

4. The method of claim 1, wherein the first medium is a television 
broadcast spectrum. 

5. The method of claim 1, wherein the first medium is a cable television 

network. 
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6. The method of claim 1, wherein the information in the broadcast signal 
to which the timing information corresponds are events in a show. 

7. The method of claim 1, wherein the second user reception device is 
capable of communicating via the Internet. 

8. The method of claim 1 , wherein the second medium is the Internet. 

9. A system for synchronizing on-line content with a broadcast, the 
system comprising: 

a receiver to receive timing information corresponding to information in a 
broadcast signal via a reception medium, the timing information corresponding to 
information in a predetermined broadcast signal; and 

a display device to generate user-perceptible indicia at times 
corresponding to the timing information, the user-perceptible indicia corresponding to 
content of the broadcast signal. 

10. The system of claim 9, wherein the receiver is capable of 
communicating via the Internet. 

1 1 . The system of claim 9, wherein the reception medium is the Internet. 
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12. The system of claim 9, wherein the display device is a personal 

computer. 

13. A client device for a sync-to-broadcast system, the device comprising: 
communication means for establishing communication between the client 

device and a communication network; 

timing reception means for receiving via the communication means timing 
information corresponding to events in a broadcast signal; and 

display means for generating user-perceptible indicia at times 
corresponding to the timing information, the user-perceptible indicia corresponding to 
content of the broadcast signal. 

14. The client device of claim 13, further comprising synchronization 
means for receiving via the communication means synchronization cues and using the 
synchronization cues to synchronize indicia generated by the display means with the 
broadcast signal. 

1 5. The client device of claim 13, further comprising: 

input means for receiving user commands relating to indicia generated by 
the display means and processing the commands; and 

local action reporting means for transmitting via the communication 
means information relating to the user commands over the communication network. 
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16. The client device of claim 1 5, wherein the information transmitted by 
the local action reporting means includes at least one of answers to questions, text 
messages, activities in which the user is involved; advertisements the user has seen; and 
login status of the user. 

17. The client device of claim 13, wherein the communication means and 
timing reception means are resident in one of a WebTV device, a set top box and an 
enhanced television. 

1 8. The client device of claim 1 3, wherein the communication means, 
timing reception means and display means are resident in one of a personal computer and 
a wireless device. 

19. A multi-user server system for connecting multiple users over the 
Internet comprising: 

communication means for establishing communication between the server 

and a communication network; 

a load manager application for communicating via the communication 
means with a plurality of client devices and partitioning each client device into one of a 
plurality of rooms; and 

activity manager means for communicating via the communication means 
with the plurality of clients to exchange data therewith, the data corresponding to 
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activities activities associated with the room to which each client device has been 
assigned. 

20. The multi-user server system of claim 19, further comprising web 
service means for serving via the communication means at least one of web pages, assets, 
software and media to the plurality of client devices based on rooms to which the client 
devices have been partitioned. 

21. The multi-user server system of claim 19, further comprising 
advertising service means for serving on-line advertising synchronized with commercials 
in the broadcast signal. 

22. The multi-user server system of claim 20, further comprising: 

at least one media library containing at least one of graphics, video, sound, 

an animation; 

wherein the web service means is further for retrieving information from 
the media library and serving it via the communication means. 

23. The multi-user server system of claim 19, further comprising: 
archiving means for receiving via the communication means information 

related to on-line activities of the plurality of client devices; and 

a database for storing and tracking the on-line activity information. 
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24. The multi-server system of claim 23, wherein the on-line activity 
information includes at least one of user profiles, interaction, scoring results and ad 
views. 

25. A control system for an on-line sync-to-broadcast show, the system 

comprising: 

communication means for establishing communication between the 
system and a communication network; 

synchronization control means for transmitting via the communication 
means information for controlling synchronization of an on-line show with a broadcast 
signal; and 

data integration control means for generating a signal representative of 
information gathered via the communication means to be integrated into the broadcast 
signal. 

26. The system of claim 25, wherein the synchronization control means 
includes an automatic synchronization control responsive to a tone imbedded in a vertical 
blanking interval of the broadcast signal to transmit via the communication means 
information to synchronize the on-line show with the broadcast. 

27. The system of claim 25, wherein the synchronization control means 
includes a manual synchronization control for transmitting via the communication-means 
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information to synchronize the on-line show with the broadcast responsive to an operator 
action at the control system. 

28. The system of claim 25, wherein the synchronization control means 
includes slip synchronization control means for transmitting via the communication 
means information to slip synchronization between the on-line show and the broadcast 
signal responsive to an operator action at the control system. 

29. The system of claim 25, further comprising flow control means for 
transmitting via the communication means information to control a flow of information 
within the on-line show. 

30. The system of claim 29, wherein the flow control means includes a 
start/stop control for transmitting information via the communication means to manually 
start and stop the on-line show responsive to an operator action at the control system. 

31. The system of claim 29, wherein the flow control means includes kill 
control means for transmitting via the communication means information to terminate a 
specific function within the on-line show. 

32. The system of claim 25, further comprising message transmission 
means for sending via the communication means a real-time message for viewers of the 
broadcast signal and on-line show. 
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33. The system of claim 25, further comprising standings control means 
for managing the information received via the communication means relating to viewers 
of the broadcast signal and on-line show. 

34. The system of claim 25, further comprising server status reporting 
means for displaying information relating to status of the system. 

35. The system of claim 25, further comprising user reporting means for 
generating reports on information received via the communication means relating to user 
usage, user profiles, scores and advertising. 

36. A show scripting system for sync-to-broadcast shows, the system 

comprising: 

communication means for establishing communication- between the 
system and a communication network; 

script generator means for receiving via the communication means 
information representative of on-line actions of users viewing an on-line show and a 
broadcast based on a medium and based on that information generating a script matching 
specific on-line actions with time codes from the medium; 

asset integration means for adding text, artwork, sounds and music to the 
on-line show script; 

editing means for modifying the scripts and assets; and 
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uploading means for transmitting the script via the communication means. 
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Fig. 4 
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Fig. 5 Time-based Flow Chart 
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